When running SAS^® Enterprise Miner™ on a Scalable Performance Data Engine for a Hadoop database, partially written files in the database can occur. Errors similar to the following are displayed:

ERROR: There was a failure to generate metadata from the Hadoop engine.

ERROR: The bridge for SAS High-Performance Analytics encountered an internal error.

ERROR: org.apache.hadoop.security.AccessControlException: Permission denied:
        user=hadoop, access=WRITE,
        inode="/data/aatest/spde/w32/src":hadoop:supergroup:dr-xr-xr-x


ERROR: Call to method org.apache.hadoop.fs.FileSystem::mkdirs failed.

The problem occurs if the diagram includes a high-performance node, and there is a termination of the MPCONNECT session for any reason (user-termination, or other). These partially written files are orphans in the database, and they can be handled by the following approaches.

• When the MPCONNECT session is terminated, there are unfinished permanent tables in the database. This problem occurs if the flow is executing either the HP Data Partition node, or the HP Text Miner node. The associated "write" lock file for the table remains. Even if the nodes are re-run, the names of these tables remain unchanged, and the nodes fail.

  To work around the problem, delete any problem node, and replace it with a new one.

• For any other HP node, when the MPCONNECT session is terminated, the temporary tables that are built are orphaned.

  To work around the problem, re-run the flow. These temporary table names are randomly generated, so when the flow is re-run, the temporary tables are deleted from the database.

For both issues, your administrator might need to delete the orphaned lock files at the operating-system level, because deleting those files is outside of the SAS system control.

Operating System and Release Information